https://github.com/GreatLearningAIML1/gl-pgp-aiml-uta-intl-may20-kv2001
Problem Statement: McCurr Consultancy – Attrition Analysis
- Univariate analysis – data types and description of the independent attributes which should include (name, meaning, range of values observed, central values (mean and median), standard deviation and quartiles, analysis of the body of distributions / tails, missing values, outliers
In [1]:
import pandas as pd # data processing
import numpy as np # linear algebra
import matplotlib.pyplot as plt # for plotting graphs
%matplotlib inline
import seaborn as sns
import warnings
warnings.filterwarnings('ignore')
In [2]:
data1 = pd.read_excel('HR_Employee_Attrition Dataset.xlsx') #import Excel data
data1.dtypes
Out[2]:
In [3]:
data1.shape
Out[3]:
In [4]:
data1.describe() #central values (mean and median), standard deviation and quartiles, etc
Out[4]:
In [5]:
data1.info()
In [6]:
data1.tail()
Out[6]:
In [7]:
data1.isnull().any() # to identify any null values
Out[7]:
In [8]:
for col in data1.columns:
print (col, len(data1[col].unique())) # number of unique in each column
CHECK FOR MISSING VALUES
In [9]:
data1.isna().sum()/(len(data1))*100
Out[9]:
In [10]:
data1.loc[:, data1.isnull().any()].columns
Out[10]:
In [11]:
data1.skew()
Out[11]:
Univarite Plots:
In [12]:
sns.distplot(data1['Age']);
In [13]:
sns.distplot(data1['DistanceFromHome']);
In [14]:
sns.distplot(data1['MonthlyIncome']);
In [15]:
sns.distplot(data1['NumCompaniesWorked']);
In [16]:
sns.distplot(data1['TotalWorkingYears']);
In [17]:
sns.distplot(data1['TrainingTimesLastYear']);
In [18]:
sns.distplot(data1['WorkLifeBalance']);
In [19]:
sns.distplot(data1['YearsAtCompany']);
In [20]:
sns.distplot(data1['YearsInCurrentRole']);
In [21]:
sns.distplot(data1['YearsSinceLastPromotion']);
In [22]:
sns.distplot(data1['YearsWithCurrManager']);
- Bi-variate analysis between the predictor variables and between the predictor variables and target column. Comment on your findings in terms of their relationship and degree of relation if any. Visualize the analysis using boxplots and pair plots, histograms or density curves.
In [23]:
# Reassign target
data1.Attrition.replace(to_replace = dict(Yes = 1, No = 0), inplace = True)
Count of Attrition
In [24]:
sns.countplot(data1['Attrition']);
In [25]:
plt.figure(figsize=(8,4))
ax = sns.barplot(y="Age", x="Attrition", data=data1)
In [26]:
plt.figure(figsize=(8,4))
ax = sns.barplot(y="DistanceFromHome", x="Attrition", data=data1)
In [27]:
plt.figure(figsize=(8,4))
ax = sns.barplot(y="MonthlyIncome", x="Attrition", data=data1)
In [28]:
plt.figure(figsize=(8,4))
ax = sns.barplot(y="NumCompaniesWorked", x="Attrition", data=data1)
In [29]:
plt.figure(figsize=(8,4))
ax = sns.barplot(y="TotalWorkingYears", x="Attrition", data=data1)
In [30]:
plt.figure(figsize=(8,4))
ax = sns.barplot(y="TrainingTimesLastYear", x="Attrition", data=data1)
In [31]:
plt.figure(figsize=(8,4))
ax = sns.barplot(y="WorkLifeBalance", x="Attrition", data=data1)
In [32]:
plt.figure(figsize=(8,4))
ax = sns.barplot(y="YearsAtCompany", x="Attrition", data=data1)
In [33]:
plt.figure(figsize=(8,4))
ax = sns.barplot(y="YearsInCurrentRole", x="Attrition", data=data1)
In [34]:
plt.figure(figsize=(8,4))
ax = sns.barplot(y="YearsSinceLastPromotion", x="Attrition", data=data1)
In [35]:
plt.figure(figsize=(8,4))
ax = sns.barplot(y="YearsWithCurrManager", x="Attrition", data=data1)
Degree of Relationship between target variable and feature: Attrition rate is high when the YearsWithCurrManager is lower(newer). Attrition rate is high when the YearsInCurrentRole is lower(newer). Attrition rate is high when the YearsAtCompany is lower(newer).Attrition rate is high when the TotalWorkingYears is lower(newer). Attrition rate is high when the NumCompaniesWorked is more. Attrition rate is high when the DistanceFromHome is higher.
In [36]:
sns.pairplot(data1)
Out[36]:
In [37]:
for i in ['Age','DistanceFromHome','MonthlyIncome','NumCompaniesWorked','PercentSalaryHike','TotalWorkingYears','TrainingTimesLastYear','WorkLifeBalance','YearsAtCompany','YearsInCurrentRole','YearsSinceLastPromotion','YearsWithCurrManager']:
sns.boxplot(x='Attrition',y=i,data=data1)
plt.show()
In [38]:
fig,ax = plt.subplots(figsize=(10,7))
sns.violinplot(x='Gender', y='MonthlyIncome',hue='Attrition',split=True,data=data1)
Out[38]:
Gender ditribution does not seems to make much difference to attrition.
In [39]:
f,ax=plt.subplots(figsize=(20,20))
sns.heatmap(data1.corr(),annot=True,linewidths=0.5,linecolor="green",fmt=".1f",ax=ax)
plt.show()
In [40]:
sns.boxplot(x='MonthlyIncome',y='JobRole',data=data1)
Out[40]:
- Feature Engineering techniques a) Identify opportunities (if any) to create a composite feature, drop a feature (if required) b) Get data model ready and do a train test split. c) Decide on complexity of the model, should it be simple linear model in terms of parameters or would a quadratic or higher degree.
In [41]:
data1.columns
Out[41]:
In [42]:
data1.drop(['EmployeeNumber', 'Over18', 'StandardHours'],axis="columns", inplace=True)
In [43]:
data1.shape
Out[43]:
In [44]:
df_left = data1[data1['Attrition'] == 1]
df_stay = data1[data1['Attrition'] == 0]
In [45]:
print(f'Percentage of employees left the Company : {(len(df_left) / len(data1))*100} %')
print(f'Percentage of employees Stayed the Company : {(len(df_stay) / len(data1))*100} %')
In [46]:
x_cat = data1.select_dtypes(include='object')
x_cat
Out[46]:
In [47]:
from sklearn.preprocessing import OneHotEncoder
onehotencoder = OneHotEncoder()
x_cat = onehotencoder.fit_transform(x_cat).toarray()
In [48]:
x_cat = pd.DataFrame(x_cat)
x_cat
Out[48]:
In [49]:
x_numerical = data1.select_dtypes(exclude='object')
x_numerical.drop('Attrition',axis=1,inplace=True)
x_numerical
Out[49]:
In [50]:
x_data = pd.concat([x_cat,x_numerical],axis=1)
x_data
Out[50]:
In [51]:
from sklearn.preprocessing import MinMaxScaler
scalar = MinMaxScaler()
x = scalar.fit_transform(x_data)
In [52]:
x
Out[52]:
In [53]:
y = data1['Attrition']
y
Out[53]:
In [54]:
from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test = train_test_split(x,y,test_size=0.25)
In [55]:
print(f'Shape of X train :{X_train.shape}')
print(f'Shape of X test :{X_test.shape}')
print(f'Shape of y train :{y_train.shape}')
print(f'Shape of y test :{y_test.shape}')
In [56]:
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
In [57]:
model = LogisticRegression()
model.fit(X_train,y_train)
Out[57]:
In [58]:
y_pred = model.predict(X_test)
In [59]:
from sklearn.metrics import confusion_matrix,classification_report
print(f'Accuracy : {100 * accuracy_score(y_pred,y_test)}')
In [60]:
cm = confusion_matrix(y_pred,y_test)
sns.heatmap(cm,annot=True,fmt='d')
Out[60]:
In [61]:
print('Classification report')
print('======='*10)
print(classification_report(y_pred,y_test))
print('======='*10)
In [62]:
from sklearn.ensemble import RandomForestClassifier
randomforest = RandomForestClassifier()
randomforest.fit(X_train,y_train)
Out[62]:
In [63]:
y_pred = randomforest.predict(X_test)
In [64]:
cm = confusion_matrix(y_pred,y_test)
sns.heatmap(cm,annot=True,fmt='d')
Out[64]:
In [65]:
print('Classification report')
print('======='*10)
print(classification_report(y_pred,y_test))
print('======='*10)
In [66]:
from xgboost import XGBClassifier
xgb = XGBClassifier()
xgb.fit(X_train,y_train)
Out[66]:
In [67]:
y_pred = xgb.predict(X_test)
In [68]:
cm = confusion_matrix(y_pred,y_test)
sns.heatmap(cm,annot=True,fmt='d')
Out[68]:
In [69]:
print('Classification report')
print('======='*10)
print(classification_report(y_pred,y_test))
print('======='*10)
In [72]:
!pip3 install -U imbalanced-learn
In [73]:
from imblearn.over_sampling import SMOTE
sm = SMOTE(random_state=27,sampling_strategy=1.0)
X_train,y_train = sm.fit_sample(X_train,y_train)
In [74]:
smote_logistic = LogisticRegression()
smote_logistic.fit(X_train,y_train)
Out[74]:
In [75]:
smote_pred = smote_logistic.predict(X_test)
In [76]:
sns.heatmap(confusion_matrix(y_test,smote_pred),annot=True,fmt='.2f',cmap='YlGnBu')
plt.show()
In [77]:
print('Classification report')
print('======='*10)
print(classification_report(smote_pred,y_test))
print('======='*10)
In [78]:
randomforest = RandomForestClassifier()
randomforest.fit(X_train,y_train)
Out[78]:
In [79]:
y_pred_smote = randomforest.predict(X_test)
In [80]:
sns.heatmap(confusion_matrix(y_pred_smote,y_test),annot=True,fmt='d',cmap='YlGnBu')
plt.show()
In [81]:
from sklearn.model_selection import KFold
from sklearn.model_selection import cross_val_score
num_folds = 50
seed = 7
kfold = KFold(n_splits=num_folds, random_state=seed)
model = LogisticRegression()
results = cross_val_score(model, X_train, y_train, cv=kfold, scoring='roc_auc')
print(results)
print("Accuracy: %.3f%% (%.3f%%)" % (results.mean()*100.0, results.std()*100.0))
Techniques employed to squeeze that extra performance out of the model without making it over fit or under fit. Use Grid Search or Random Search on any of the two models used above. Make a DataFrame to compare models after hyperparameter tuning and their metrics as above.
In [82]:
from sklearn.model_selection import RandomizedSearchCV
from scipy.stats import randint as sp_randint
In [83]:
# build a classifier
clf = RandomForestClassifier(n_estimators=50)
In [84]:
# specify parameters and distributions to sample from
param_dist = {"max_depth": [3, None],
"max_features": sp_randint(1, 11),
"min_samples_split": sp_randint(2, 11),
"min_samples_leaf": sp_randint(1, 11),
"bootstrap": [True, False],
"criterion": ["gini", "entropy"]}
In [85]:
# run randomized search
samples = 10 # number of random samples
randomCV = RandomizedSearchCV(clf, param_distributions=param_dist, n_iter=samples) #default cv = 3
In [86]:
randomCV.fit(X_train, y_train)
print(randomCV.best_params_)
In [ ]: